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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.114. Applicant's submission filed on April 4 2007 has been entered. 

Response to Arguments 

Applicant's arguments have been considered but are moot in view of the new ground(s) 
of rejection. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

Claims 1-3, 5-10 and 32-35 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Szeliski et al. US patent no. 6,157,747 in view of Luken US patent no. 5,923,334 and 
further in view of Katayama et al. US Pub. no. 2002/0081019 Al . 

-Regarding claim 1, Szeliski et al. teaches a computerized method for creating a three 
dimensional model from one or more image panoramas (see column 1, lines 6-11, column 4, 
lines 8-9 and 66-67): receiving one or more image panoramas representing a visual scene and 
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having one or more objects (see fig. 6, column 5, lines 51-52); a directional vector indicating an 
orientation of the visual scene with respect to a reference coordinate system (see fig. 15, column 
22, lines 4-9), creating a three dimensional model of the visual scene from the transformed image 
panoramas using the reference coordinate system and comprising geometry information 
describing the one or more objects contained in the scene (see column 5, lines 7-35, column 9, 
lines 4-26), transforming the image panoramas such that the directional vectors are substantially 
aligned relative to the reference coordinate system (see figs. 11 and 15, column 21, lines 25-59, 
column 22, lines 2-35), and aligning the transformed image panoramas to each other (see figs. 1 
and 2B, column 9, lines 29-36). 

Szeliski does not directly teach a method for determining a directional vector for each 
image panorama. However in similar art Luken teaches said method (see figs. 7-10, 14 and 17, 
column 12, lines 35-44). 

Therefore it would have been obvious to one skilled in the art to combine Szeliski with 
Luken, because it would provide an efficient system for generating and viewing three- 
dimensional panoramic images based environment maps, and offer an improved level of 
interactive graphical feedback (see column 3, lines 5-8). 

Szeliski and Luken do not directly teach a method wherein creating a three dimensional 
model includes identifying a selected object in the transformed and aligned image panoramas 
and associating geometry information with the selected object, the geometry information 
comprising 3-D coordinates describing the position and orientation of the selected object in the 
reference coordinate system . 
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However in similar art Katayama et al. teaches said method (see figs. 1, 3 and 5B, [0082], 
[0083], [0089]). Therefore it would have been obvious to one skilled in the art at the time of the 
invention to combine Katayama with Szeliski and Luken, because it would be used to modify 
objects within the three-dimensional panoramic view (see [0154], [0189], [0350]). 

-Regarding claim 2, Szeliski teaches a method wherein the directional vector is 
determined based, at least in part, on instructions identifying elements of the image panorama 
received from a user (see column 8, lines 30-32, column 27, lines 64-66). 

-Regarding claim 3, Szeliski teaches a method wherein the instructions from the user 
identify two or more substantially parallel features in the image (see column 20, line 64-column 
21, line 6 and lines 25-34). 

-Regarding claim 5, Szeliski teaches a method wherein the instructions from the user 
identifying a horizon line of the image panorama (see fig 4, column 9, lines 54-62). 

-Regarding claim 6, Szeliski teaches a method wherein the instructions comprise the 
identification of two or more areas of the image (see figs. 6 and 18, column 20, lines 49-50, 
column 27, lines 32-44), each area containing one or more elements and further comprising 
automatically identifying the two elements contained in the two or more areas (see fig. 6, column 
20, lines 49-50, column 32, lines 24-27). 

-Regarding claim 7, Szeliski teaches a method further comprising using edge detection to 
automatically identify two elements (see fig. 32, column 30, lines 13-17). 
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-Regarding claim 8, Szeliski teaches a method wherein the image panoramas are aligned 
relative to the reference coordinate system such that the directional vector is at least substantially 
parallel to one axis of the reference coordinate system (see figs. 1 1 and 15, column 21, lines 25- 
59, column 22, lines 2-35 [it is obvious that as the image rotates into alignment the directional 
vector will be parallel to one axis]). 

-Regarding claim 9, Szeliski teaches a method wherein the image panoramas are aligned 
relative to the reference coordinate system such that the directional vector is at least substantially 
orthogonal to one axis of the reference coordinate system (see figs. 11 and 15, column 21, lines 
25-59, column 22, lines 2-35 [it is obvious that as the image rotates into alignment the directional 
vector will be orthogonal to one axis]). 

-Regarding claim 10, Szeliski teaches a method wherein the image panoramas are aligned 
according to instructions received from a user (see column 8, lines 30-32, column 25, lines 64- 
66, column 27, lines 64-66). 

-Regarding claim 32, Szeliski et al. teaches a system for creating a three dimensional 
model from one or more image panoramas (see fig. 2B, column 4, lines 66-67, column 9, lines 4- 
26) comprising: means for receiving one or more image panoramas representing a visual scene 
having one or more objects (see column 4, lines 8-13 [captured = received images] and 66-67); 
means for allowing a user to interact with the system (see column 8, lines 30-32, column 25, 
lines 64-66, column 27, lines 64-66); means for aligning the image panoramas relative to each 
other (see figs. 1, 2B, 1 1 and 15, column 9, lines 29-36, column 21, lines 25-59, column 22, lines 
2-35); and means for creating a three dimensional model from the aligned panoramas (see figs. 1 
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and 2B, column 9, lines 29-36), the model comprising geometry information describing the one 
or more objects contained in the scene (see column 5, lines 7-35, column 9, lines 4-26) 

Szeliski does not directly teach a method for determining a directional vector for each 
image panorama. However in similar art Luken teaches said method (see figs. 7-10, 14 and 17, 
column 12, lines 35-44). 

Therefore it would have been obvious to one skilled in the art to combine Szeliski with 
Luken, because it would provide an efficient system for generating and viewing three- 
dimensional panoramic images based environment maps, and offer an improved level of 
interactive graphical feedback (see column 3, lines 5-8). 

-Regarding claim 33, Szeliski teaches a system wherein the input images comprise two- 
dimensional images (see figs. 1 and 2B, [I^ are two dimensional]). 

-Regarding claim 34, Szeliski teaches a system wherein the input images comprise three- 
dimensional images including geometry information (see fig. 2B, column 5, lines 7-35, column 
9, lines 4-26). 

Regarding claim 35, Szeliski teaches a system wherein the image panoramas are aligned 
according to instructions received from a user (see figs. 1 and 2B, column 8, lines 30-32, column 
25, lines 64-66, column 27, lines 64-66). 

Claims 11, 12, 22-28 and 36 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Szeliski et al. US patent no. 6,157,747 in view of Katayama et al. US Pub. no. 
2002/0081019 Al. 
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-Regarding claim 11, Szeliski et al. teaches a computerized method of interactively 
editing objects in a panoramic image (see column 1, lines 6-11, column 4, lines 8-9 and 66-67, 
column 8, lines 30-32, column 27, lines 64-66) comprising: receiving an image panorama 
representing a visual scene (see fig. 3, column 5, lines 46-47, column 9, lines 55-62), the image 
panorama having one or more objects and a point source (see figs. 3, 4 and 6, column 5, lines 46- 
52); creating a three dimensional model of the visual scene using features of the visual scene and 
the point source (see fig. 2B, column 4, lines 66-67, column 9, lines 4-26); receiving an edit to 
one or more of the objects in the panorama (see fig. 1, column 7, lines 1 1-20); transforming the 
edit relative to a viewpoint defined by the point source (see column 10, lines 27-38, column 11, 
lineslO-21); and projecting the transformed edit onto the objects (see figs 2B-8, column 3, lines 
58-61, column 31, lines 44-51). 

Szeliski does not directly teach a method wherein creating a three dimensional model 
includes identifying a selected object in the image panorama and associating geometry 
information with the selected object, the geometry information comprising 3-D coordinates 
describing the position and orientation of the selected object in the reference coordinate system . 

However in similar art Katayama et al. teaches said method (see figs. 1, 3 and 5B, [0082], 
[0083], [0089]). Therefore it would have been obvious to one skilled in the art at the time of the 
invention to combine Katayama with Szeliski and Luken, because it would be used to modify 
objects within the three-dimensional panoramic view (see [0154], [0189], [0350]). 
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-Regarding claim 12, Szeliski teaches a three-dimensional model (see figs. 2B, 27, 30 and 
32, column 28, lines 29-31, column 29, lines 58-61), three-dimensional models inherently 
contain one or more of depth information and geometry information. 

-Regarding claim 22, Szeliski et al. teaches a method for projecting texture information 
onto a geometric feature within an image panorama (see fig. 2B, column 3, lines 58-61, column 
27, lines 34-55) comprising: receiving instructions from a user identifying a three-dimensional 
geometric surface within an image panorama (see column 7, lines 10-20), the image panorama 
containing features having one or more textures (see column 5, lines 27-35); determining a 
directional vector from the three-dimensional geometric surface (see figs. 2B and 15, column 22, 
lines 4-31); creating a geometric model of the image panorama based at least in part on the three- 
dimensional geometric surface and the directional vector (see figs. 2A-B, column 7, lines 25-33, 
column 9, lines 4-26, column 22, lines 4-31); and applying the one or more textures to the 
features in the image panorama based on the geometric model (see fig. 2B, column 9, lines 29- 
31, column 32, lines 22-29 and 41-43). 

Szeliski does not directly teach a method wherein creating a three dimensional model 
includes associating geometry information with the selected feature, the geometry information 
comprising 3-D coordinates describing the position and orientation of the selected feature in the 
reference coordinate system . 

However in similar art Katayama et al. teaches said method (see figs. 1, 3 and 5B, [0082], 
[0083], [0089]). Therefore it would have been obvious to one skilled in the art at the time of the 
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invention to combine Katayama with Szeliski and Luken, because it would be used to modify 
objects within the three-dimensional panoramic view (see [0154], [0189], [0350]). 

-Regarding claim 23, Szeliski teaches a method wherein the instructions are received 
using an interactive drawing tool (see column 27, line 64-colum 28, line 2). 

-Regarding claim 24, Szeliski teaches a method wherein the three-dimensional geometric 
surface is one of a floor, a wall, or a ceiling (see figs. 33 and 38, column 30, lines 44-45). 

-Regarding claim 25, Szeliski teaches a method wherein the directional vector is 
orthogonal to the planar surface (see fig. 15, column 15, lines 64-66, column 22, lines 4-6). 

Regarding claim 26, Szeliski teaches a method wherein the geometric model comprises 
depth information (see figs. 2B, 27 and 30, column 28, lines 29-33). 

-Regarding claim 27, Szeliski teaches a method wherein the texture information 
comprises color information (see column 28, lines 13-18). 

-Regarding claim 28, Szeliski teaches a method comprising environment based texture 
mapping (see fig. 2B, column 5, lines 27-30, column 9, lines 12-26), in said method the texture 
information inherently includes luminance information (see figs. 33-38). 

-Regarding claim 36, Szeliski et al. teaches a system for interactively editing objects in a 
panoramic image (see column 1, lines 6-11, column 4, lines 8-9 and 66-67, column 8, lines 30- 
32, column 27, lines 64-66) comprising: a receiver for receiving one or more image panoramas 
representing a visual scene (see fig. 3, column 5, lines 46-47, column 9, lines 55-62) having one 
or more objects and a point source (see figs. 3, 4 and 6, column 5, lines 46-52); a modeling 
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module for creating a three dimensional model of the visual scene (see figs. 2B, 27, 30 and 32, 
column 4, lines 66-67, column 9, lines 4-26, column 28, lines 29-31, column 29, lines 58-61) 
including depth information describing the objects (three-dimensional models inherently contain 
depth information), one or more interactive editing tools for providing an edit to one or more 
objects in the panorama (see fig. 1, column 7, • lines 11-20, column 8, lines 30-32); a 
transformation module for transforming the edit relative to a viewpoint defined by the point 
source (see figs. 2B and 8, column 10, lines 27-38, column 11, lineslO-21); and a rendering 
module for projecting the transformed edit onto the objects (see figs. 1 and 2A-B, column 3, 
lines 58-61, column 31, lines 44-51). 

Szeliski does not directly teach a method wherein creating a three dimensional model 
includes identifying a selected object in the image panorama and associating geometry 
information with the selected object, the geometry information comprising 3-D coordinates 
describing the position and orientation of the selected object in the reference coordinate system . 

However in similar art Katayama et al. teaches said method (see figs. 1, 3 and 5B, [0082], 
[0083], [0089]). Therefore it would have been obvious to one skilled in the art at the time of the 
invention to combine Katayama with Szeliski and Luken, because it would be used to modify 
objects within the three-dimensional panoramic view (see [0154], [0189], [0350]). 

Claims 13-21 and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Szeliski et al. US patent no. 6,157,747 view of Katayama et al. US Pub. no. 2002/0081019 Al 
and further in view of Blank US patent no. 5,469,536.13. 
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-Regarding claim 13, Szeliski and Katayama do not directly teach a method comprising 
receiving an edit to color information associated with the objects of the image. However in 
similar art Blank teaches said method (see column 6, lines 29-47). Therefore it would have been 
obvious to one skilled in the art to combine Szeliski and Katayama with Blank, because it would 
enable the user to quickly and efficiently modify or enhance the appearance of an image to 
desired goal (see column 6, lines 23-28). 

-Regarding claim 14, Szeliski and Katayama do not directly teach a method comprising 
receiving an edit to alpha information associated with the objects of the image. However in 
similar art Blank teaches said method (see fig. 11, column 20, lines 61-66). Therefore it would 
have been obvious to one skilled in the art to combine Szeliski and Katayama with Blank, 
because it would enable the user to quickly and efficiently modify or enhance the appearance of 
an image to desired goal (see column 6, lines 23-28). 

-Regarding claim 15, Szeliski and Katayama do not directly teach a method comprising 
receiving an edit to depth information associated with the objects of the image. However in 
similar art Blank teaches said method (see column 3, lines 59-67, column 13, lines 8-16). 
Therefore it would have been obvious to one skilled in the art to combine Szeliski and Katayama 
with Blank, because it would enable the user to quickly and efficiently modify or enhance the 
appearance of an image to desired goal (see column 6, lines 23-28). 

-Regarding claim 16, Szeliski and Katayama do not directly teach a method comprising 
receiving an edit to geometry information associated with the objects of the image. However in 
similar art Blank teaches said method (see column 47, lines 1 1-20). Therefore it would have been 
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obvious to one skilled in the art to combine Szeliski and Katayama with Blank, because it would 
enable the user to quickly and efficiently modify or enhance the appearance of an image to 
desired goal (see column 6, lines 23-28). 

-Regarding claim 17, Szeliski and Katayama do not directly teach a method comprising: 
providing a user with an interactive drawing tool that specifies edits for one or more objects of 
the image; and receiving the edits made by the user using the interactive drawing tool. However 
in similar art Blank teaches said method (see column 21, lines 18-23, column 28, lines 28-35). 
Therefore it would have been obvious to one skilled in the art to combine Szeliski and Katayama 
with Blank, because it would enable the user to quickly and efficiently modify or enhance the 
appearance of an image to desired goal (see column 6, lines 23-28). 

-Regarding claims 18 and 37, Szeliski and Katayama do not directly teach a method 
wherein the interactive drawing tool (editing tool) is one of an extrusion tool, a ground plane 
tool, a depth chisel tool or a non-uniform rational B-spline tool. However in similar art Blank 
teaches said method (see column 8, lines 46-52, column 34, lines 22-27). Therefore it would 
have been obvious to one skilled in the art to combine Szeliski and Katayama with Blank, 
because it would enable the user to quickly and efficiently modify or enhance the appearance of 
an image to desired goal (see column 6, lines 23-28). 

-Regarding claim 19, Szeliski and Katayama do not directly teach a method wherein the 
interactive drawing tool specifies a selected value for depth for objects of the image. However in 
similar art Blank teaches said method (see column 21, lines 14-27, column 22, lines 35-62). 
Therefore it would have been obvious to one skilled in the art to combine Szeliski and Katayama 
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with Blank, because it would enable the user to quickly and efficiently modify or enhance the 
appearance of an image to desired goal (see column 6, lines 23-28). 

-Regarding claim 20, Szeliski and Katayama do not directly teach a method wherein the 
interactive drawing tool incrementally adds to the depth for objects of the image. However in 
similar art Blank teaches said method (see column 34, lines 8-11). Therefore it would have been 
obvious to one skilled in the art to combine Szeliski and Katayama with Blank, because it would 
enable the user to quickly and efficiently modify or enhance the appearance of an image to 
desired goal (see column 6, lines 23-28, column 48, lines 26-39). 

-Regarding claim 21, Szeliski and Katayama do not directly teach a method wherein the 
interactive drawing tool incrementally subtracts from the depth for objects of the image. 
However in similar art Blank teaches said method (see column 34, lines 22-26). Therefore it 
would have been obvious to one skilled in the art to combine Szeliski and Katayama with Blank, 
because it would enable the user to quickly and efficiently modify or enhance the appearance of 
an image to desired goal (see column 6, lines 23-28, column 48, lines 26-39). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Enrique L Santiago whose telephone number is (571) 272-7648. 
The examiner can normally be reached on Monday to Thursday from 6:30 A.M. to 4:30 P.M. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mark K. Zimmerman whose telephone number is (571) 272-7653, can be reached on 
Monday to Friday from 7:00 A.M. to 3:30 P.M. 
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Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 

Washington, D.C. 20231 
Faxed to: 571-273-8300 

Hand carried deliveries must be made to the Customer Service Window (now located at 
the Randolph Building, 401 Dulany Street, Alexandria, VA 22314). 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
Enrique L. Santiago 
May 14, 2007 

MARK ZIMMERMAN 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 




